Journal of Soft Computing in Civil Engineering 6-3 (2022) 39-62 


van 
ws 


ouyan 
Ores 


Journal of Soft Computing in Civil Engineering 


Journal homepage: www.jsoftcivil.com 


Contents lists available at SCCE 


sCC@ 


Parametric Assessment of Concrete Constituent Materials Using 
Machine Learning Techniques 


Yazan Alzubi'*®, Jamal Al Adwan’, Ahmed Khatatbeh’, Buthainah Al-kharabsheh” 
1. Civil Engineering Department, Faculty of Engineering Technology Al-Balga Applied University, 11134 Amman, 


Jordan 


2. Civil Engineering Department, Faculty of Engineering, Al Albayt University, 25113 Mafraq, Jordan 
Corresponding author: yazan.alzubi@ bau.edu.jo 


€&) _ https://doi.org/10.22115/SCCE.2022.334664.1417 


ARTICLE INFO 


ABSTRACT 


Article history: 

Received: 18 March 2022 
Revised: 01 July 2022 
Accepted: 14 July 2022 


Keywords: 

Concrete materials 
management; 

Compressive strength; 
Partial dependence analysis; 
Machine learning; 


Feature importance analysis. 


Nowadays, technology has advanced, particularly in machine 
learning which is vital for minimizing the amount of human 
work required. Using machine learning approaches to estimate 
concrete properties has unquestionably triggered the interest of 
many researchers across the globe. Currently, an assessment 
method is widely adopted to calculate the impact of each input 
parameter on the output of a machine learning model. This 
paper evaluates the capability of various machine learning 
methodologies in conducting parametric assessments to 
understand the influence of each concrete constituent material 
on its compressive strength. It is accomplished by conducting a 
partial dependence analysis to quantify the effect of input 
features on the prediction results. As a part of the study, the 
effects of machine learning method selection for such analysis 
are also investigated by employing a concrete compressive 
strength algorithm developed using a decision tree, random 
forest, adaptive boosting, stochastic gradient boosting, and 
extreme gradient boosting. Additionally, the significance of the 
input features to the accuracy of the constructed estimation 
models is ranked through drop-out loss and MSE _ reduction. 
This investigation shows that the machine learning techniques 
could accurately predict the concrete's compressive strength 
with very high performance. Further, most analyzed algorithms 
yielded similar estimations regarding the strength of concrete 
constituent materials. In general, the study's results have shown 
that the drop-out loss and MSE _ reduction outputs were 
misleading, whereas the partial dependence plots provide a clear 
idea about the influence of the value of each feature on the 
prediction outcomes. 
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1. Introduction 


In the construction sector, there is no question that concrete is the most widely used material in 
the world [1,2]. In practice, the compressive strength of concrete is one of the most important 
factors to consider when it comes to mixture design and optimization, but getting it at mature age 
involves a lengthy experimental procedure [3,4]. However, mathematical modeling of this 
property using traditional approaches is deemed a significantly challenging task due to the 
nonlinear nature of the relation between the concrete's constituents and its characteristics. 
Recently, several studies have handled estimating the compressive strength of concrete to prove 
that machine learning-based modeling can substitute conventional methods [5—10]. Topcu & 
Saridemir [11] illustrated the capability of fuzzy logic for predicting the strength capacity of 
concrete containing fly ash. Chopra et al. [12] utilized a genetic programming approach for the 
same purpose. Moreover, Lee [13] and Habib & Yildirim [14] used artificial neural networks to 
estimate concrete characteristics. Barkhordari et al. [15] adopted ensemble machine learning 
models to estimate flyrock due to quarry blasting. The applications of bagging and boosting 
machine learning algorithms in simulating the properties of concrete were investigated by [16— 
18]. 


Indeed, machine learning techniques are very effective in estimating concrete properties at 
various maturity ages, and they can be employed to conduct detailed parametric assessments by 
evaluating the interaction and correlation between each input variable and the outcome. 
Nevertheless, their abilities to quantify the percentage of the impact that each constituent 
material has on the compressive strength of a concrete mixture, as well as to provide a detailed 
understanding of the influence of each constituent material's content on the output of the 
estimation model by using feature importance and partial dependence analyses have rarely been 
discussed in the literature. For instance, Jha et al. [19] evaluated the factors affecting concrete 
strength using feature importance analysis for different machine learning models. Anysz et al. 
[20] discussed the capabilities of explicable artificial intelligence methods to assess factors 
influencing the compressive strength of cement stabilized rammed earth. Ly et al. [21] developed 
partial dependence plots for rubberized concrete's compressive strength using deep neural 
network models. Su et al. [22] performed the partial dependence analysis for the compressive 
strength of slag-metakaolin geopolymer pastes using multivariate polynomial regression models. 
Dao et al. [23] conducted a sensitivity analysis for the compressive strength of foamed concrete 
utilizing multiple partial dependence plots based on conventional artificial neural networks. 
Mane et al. [24] developed an artificial neural network model to predict the flexural strength of 
concrete mixtures with pozzolanic materials. Nayak et al. [25] utilized an extreme learning 
machine to predict the compressive strength of concrete mixtures. Pandey et al. [26] adopted 
various machine learning approaches to design concrete mixtures with and without plasticizer. 
Naderpour et al. [27] developed artificial neural network models to predict the shear strength of 
reinforced concrete shear walls. 


Thus, it can be seen that multiple studies have adopted feature importance analysis to conduct a 
parametric assessment for concrete constituent materials, although partial dependence analysis is 
more sensitive for such applications. Moreover, it can be seen that the literature is still lacking a 
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study that evaluates and compares the capability of various machine learning models in reporting 
the effects of concrete's mixture proportioning on its compressive strength through partial 
dependence analysis. Accordingly, the primary objective of this contribution is to evaluate the 
influence of machine learning model selection on the results of the feature significance and 
partial dependency analyses. A key component is developing and evaluating several prediction 
techniques involving a decision tree, random forest, adaptive boosting, stochastic gradient 
boosting, and extreme gradient boosting. Hence, the investigations of feature importance and 
partial dependence analyses are carried out, and their findings are compared against each other. 
On the other hand, the limitation of the scope of the study is in the types of machine learning 
algorithms adopted for the investigations since it will mainly focus on commonly utilized models 
and will not go into detail about the current state-of-the-art approaches, thus will leave this point 
for future studies. Another limitation of the study is conducting a sensitivity analysis for which a 
method such as the cheap-to-evaluate uncertainty-aware global sensitivity analysis discussed by 
Amini et al. [28] can be utilized. 


2. Materials and methods 


Cement-based materials have traditionally been designed and characterized by experiments for 
decades [29]. As a result, significant discoveries were achieved through emerging modern 
computational approaches in construction materials science [30]. Currently, there is rising 
attention to scaling up the commonly utilized machine learning algorithms for classification, 
regression, clustering, or dimensionality to decrease tasks of massive datasets [31]. Indeed, 
machine learning is a powerful tool in artificial intelligence that combines statistics and 
computer science to develop more effective models that rely on training data to achieve a 
specific activity [32,33]. It is a fast-developing field that enables computer systems to gain 
knowledge directly from data and experience without programming [34]. The key target is to 
build a model that estimates the required parameter value by learning machine techniques 
inferred from the data characteristics and then using these models to perform feature importance 
and partial dependence analyses. Figure | represents the general methodology used in this study. 


2.1. Machine learning algorithms 


The decision tree (DT) regressor may be described according to the principle of divide and 
conquer to identify characteristics and model the relationships between them in big data [35]. 
The term comes from a tree structure in which the dataset (root node) is repeatedly divided into 
smaller subsets based on specific values of this property, and the initial group of tree branches is 
generated [36-38]. In practice, a training dataset is utilized to create a decision tree, and if all 
objects have the same decision class, the tree will only get a single node (leaf node) with the 
suitable decision, as described in Figure 2. On the other side, a separate feature is chosen whose 
value belongs to at least two distinct decision classes, and a collection of objects is partitioned 
into categories based on the value of the picked feature in order to construct a test node in a 
growing decision tree [39]. Every branch from the test node is followed by a series of prompting 
approaches that are performed on the remaining objects in terms of split until a leaf is attained 
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depicting the decision. There are many types of decision tree algorithms; however, the 
classification and regression tree (CART) will be used in this research owing to its capabilities. 


° Problem formation 


) . Model development 
= e Performance assessment 


| ° Findings & conclusions > 


Fig. 1. General descriptive illustration of the research methodology. 
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Fig. 2. Illustrative concept of decision tree (reproduced from Janikow [40]). 
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With an n-dimensional sample of training data {(X,,¥)s aims (x,.Y,)} <xxR and the space of 


input patterns y, the decision tree technique splits the attributes in a recursive procedure resulting 
in the output being assembled together based on their similarity. At a particular node m, the data 
is defined by Qn with Nm samples, divided into two subsets Q'""(0), and Q™* (0) in Eq. 1 and 2, 


where each candidate split 0 = (j, t;,) includes aj feature and t,, threshold. 


Qn ()={(%y) |x, St} (1) 
Qi (8) = Q,,/Qa (8) (2) 


Using the impurity function HQ) in Eq. 3, it is possible to determine the quality of a candidate 
split of node m. 
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G(Q,,-8) = a 


H(Qi"(6)) (3) 
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Hence, from Eq. 4, the parameters that reduce the impurity are picked, and the procedure is 


repeated for Qi" (8) and Qi" (8) until the tolerance depth N,, < min,,,,,,.,0r N,, = 1. 
0 =arg min, G(Q,,,8) (4) 


A random forest (RF) is a massive number of single decision trees that serve as an ensemble 
where each unique tree votes to predict the most common class as the output of the random 
forest [41,42]. As the ensemble decreases the instability related to the building of individual 
decision trees, the random forest can deal with high-dimensional datasets and complicated 
relationships and frequently produces precise models [43,44]. The decision tree model has a 
single tree, but a random forest is made up of several trees, which is the most significant 
conceptual difference between them. The training dataset is randomly sampled from the original 
data to develop the algorithm, and any ignored data is specified as out-of-bag. The random forest 
has become more prevalent in the civil engineering field during the past several years, mainly to 
deliver practical models. As a result, it is implemented by creating many CARTs and integrating 
bootstrap and aggregation concepts [45]. 


The extremely randomized trees (ERT) technique randomizes choosing splits in a tree's nodes 
[46]. The ERT approach differs from the random forest algorithm in that the thresholds for each 
candidate's feature are picked randomly, and the best of them is used as the splitting criterion 
rather than searching for the most distinctive thresholds. The significance of this fact lies in its 
capacity to reduce model variance while slightly raising the bias. 


Freund and Schapire [47] introduced the concept of adaptive boosting (AdaBoost), which was 
designed to fit the original dataset utilizing the defined regressor. The method serves by creating 
new copies of the regression algorithm employing the same training dataset while modifying the 
model weights based on the results of the previous trial. The implantation of this algorithm is 
conducted in relevance to Drucker [48]. An n-dimensional training dataset (specified before) is 
used in AdaBoost's formula, and the error e; for the whole dataset is produced by training a base 
estimator (weak learner) f(x). Hence, a series of weak learners is built and integrated to construct 
a strong model H(x) using the technique described in Eq. 5. 


H(x) = Dapp 4) g(x) (5) 


k-l k 


In this equation, v is the learning rate, a, denotes the weight of the base estimators calculated 


from Eq. 6, and g(x) represents the median of all a,f, (x) : 


eemee (6) 


Like adaptive boosting, stochastic gradient boosting (SGBoost) creates a new model ensemble to 
correct the residual error of the existing one but with a substantial variation that relies on 
reducing an objective function. In contrast to the regressor in the AdaBoost algorithm, the fragile 
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predictor in the gradient boosting method is a bigger decision tree with numerous layers. For the 
training dataset (defined above), the gradient boosting estimator y, with x; input variable is 
specified by Eq. 7. Gradient boosting is generally a greedy procedure, as stated in Eq. 8. 


y; =F, (x)= 2ihn (%:) (7) 


Fy () =F (x)+ "Ea? LL Fas (%:)+H(%,)] ®) 


M is the total number of estimators delivered in the algorithm, h,, refers to a weak learner, h(x) is 
the base predictor, and L (-) is the loss function with the negative gradient identified in Eq. 9. 


7 éL| y,,F,.(x) | 
eee (9) 


eXtreme gradient boosting (XGBoost) is a machine learning approach that is both efficient and 
scalable, and it is used for tree boosting [49]. In general, both gradients boosting and XGBoost 
adhere to the gradient boosting concept, except that XGBoost utilizes a more regularized model 
to govern over-fitting cases to obtain better outcomes. The XGBoost method employs the exact 
greedy tree approach, and the expression in Eq. 10 denotes the goal function of this method. 


obj = YLI§,.¥,]+ Lol) (10) 


i=l 


Where L (-) denotes the loss function of the model's bias, and @ is a regular term applied for 
suppressing the algorithm's complexity. 


2.2. Feature importance and partial dependence analyses 


Indeed, model interpretation is a significant step that guides the development of various models 
and decision-making strategies. It can also construct trust between the trained algorithm and 
users [50]. As mentioned previously, developed machine learning models are used in performing 
parametric assessments and reporting the influence of each input feature on the compressive 
strength of concrete. Feature importance and partial dependence analyses were adopted for 
multiple studies within this context. However, the latter is more sensitive for such applications. 


Feature importance analysis is an approach commonly adopted for the direct ranking of each 
feature on the final prediction and has three methods: drop-out loss, MSE reduction, and 
accumulated dependency. The Drop-out loss evaluates the feature importance of a specific set of 
input variables by first computing the loss in the model with and without the input parameter and 
then finding the difference between these errors [51]. In reality, a high value of drop-out loss 
implies that the variable has a considerable influence on the model's behavior, which afterward 
evaluates the variable's true implications on the target, indicating that the model is well fitted. 
The negative value of drop-out loss reveals that the existing variable within the dataset reduces 
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the model accuracy and should be eliminated. More to the point, whenever there are variables 
with a large correlation size, the feature importance propagates asymmetrically, accordingly 
causing the measure to be unreliable. 


The mean squared error (MSE) reduction is a procedure to determine the most critical 
characteristic of the dataset [52]. In general, this approach is performed by the tree traversed. 
When the outcomes of a particular node are dependent on the specified input variable, its error 
drop is multiplied by the number of samples routed to that node to its feature importance is 
added. The error decrease is computed by subtracting the MSE of data routed to that node from 
the MSE of its child nodes. In a multi-tree model, the resulting feature importance is an average 
of the feature importance of the developed trees. Generally, the feature importance obtained 
using the MSE reduction approach is sometimes misleading, especially in high cardinality 
features (many unique values) [53]. 


The cumulative effect, commonly called cumulative dependency, shows how much a given 
variable influences the model's mean expectation [54]. It is possible to emulate what the 
algorithm would estimate for a specific dataset if the value of the assigned feature varies (this 
process is called Ceteris Paribus). If the samples and their simulations are averaged, partial 
dependency for the selected feature is detected. Indeed, partial dependency is inconsistent and 
susceptible to linear correlation in some instances; hence a weighted mean should be used with 
weights corresponding to the inverse of the distances between the original samples. The 
accumulated dependency outcomes are similar to the partial dependence analysis. In this 
investigation, two methods of feature importance assessment are utilized and compared: the 
drop-out loss with 100 permutations and the MSE reduction to evaluate the significance of the 
predictors. 


Typically, feature importance analysis determines whether or not the features are necessary for 
producing the model to be successful. On the other hand, it cannot indicate how the feature's 
value impacts the prediction results. Hence, partial dependency analysis is usually performed to 
evaluate this point. The partial dependency diagram depicts the feature's marginal influence on 
the estimated result of machine learning techniques and reveals whether the interaction between 
the target and a feature is linear, monotonic, or more complicated. The partial function shows the 
average limit impact on estimating a particular feature value. It may provide answers to concerns 
about how the model's prediction results differ for a feature, allowing the influence of the 
predictor to be clearly expressed. In the beginning, the partial dependency of a specific input x is 
derived by first substituting its values with a constant one, such as x;. Then it is required to train 
the model and evaluate how much the prediction results have changed. After that, progressively 
modifying x; shows how the estimation results change depending on the investigated feature. 
Notably, one of the fundamental assumptions behind this form of analysis is that features are 
uncorrelated with one another. If this presumption is broken, the averages computed for the 
partial dependency graph will contain data points that are improbable or even impossible. 
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2.3. Dataset acquisition 


This study intends to conduct a sensitivity assessment of concrete compressive strength using 
machine learning techniques to extract the model's cause and effect relationship between the 
inputs and outputs. In order to carry out the research, a dataset with a large sample size is 
utilized, and the descriptive statistics of the developed dataset are shown in Figure 3. This dataset 
has been used previously in various research studies [55—58]. It consists of 1005 mixtures with 
their corresponding compressive strengths. The constituent materials of each concrete specimen 
were adopted as an input feature of the machine learning techniques, while the compressive 
strength capacity was selected as the outcome of the prediction model. 


us 
a 


B Sates 
“ae *. 
“ ae : 
: one 
° i 
soe 
ae” 
ceteris ooo 


?, 
ch 
-, 
3 
~ 
H 


3 
at * 
: 


® 


spy P ccc ce * 


200 900 1000 1100 1200 600 1000 0 100 200 300 400 O 25 SO 75 100 
Coarse Aggregate ‘Age te 


200 
Fine Aggregate 


Fig. 3. Visual descriptive statistics of the utilized dataset. 
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2.4. Model development and hyperparameters tunning 


The behavior of the machine learning technique is highly impacted by the selected 
hyperparameter values that should be tuned. In this study, the grid search technique with k-fold 
cross-validation was applied in the training stage to optimize the hyperparameters of the 
methods. As a result, the suggested methodology for building machine learning algorithms 
(Figure 4) initially begins by splitting the dataset into two groups: 70% training and 30% testing 
data. A cross-validation approach with 10-folds repetitions is conducted to ensure a suitable 
parameter choice. Once the hyperparameters of each approach are determined, the final tuned 
model performance is assessed by comparing the results of various scoring parameters on the test 
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Fig. 4. Illustrative description for constructing machine learning model. 
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2.5. Quality assessment 


Statistical measures and visual representations are adopted to analyze the performance of 
machine learning techniques. The goodness-of-fit is checked using the coefficient of 
determination, Eq. 11. The root mean square error (RMSE), Eq. 12, and mean absolute error 
(MAE), Eq. 13, were used for the error analysis. 


2-1-2) (11) 
> (*: -*) 
: 1/2 
rmse=|15°(x,-y,) (12) 


MAE =—)"ly,-y, (13) 


i=l 


x; 1s the measured value, x, is the mean of the measured values, y; is the predicted value y, is the 


mean of the predicted values, n is the number of observations, and m2) is the number of samples 
with a measured to the predicted ratio between 0.8 and 1.20. 


3. Results and discussions 


3.1. Estimation performance of the investigated models 


Indeed, the reliability of the decision on which input is more significant on the outcome of the 
machine learning technique is highly affected by the accuracy of the developed estimation 
model. Accordingly, measuring the performance of the investigated methods in predicting the 
compressive strength of concrete is considered essential. In general, the results of the estimations 
are shown in Figure 5 for both the training and testing stages. As seen there, the DT model was 
capable of reaching a significantly high accuracy in the training case. Whereas its performance 
dropped significantly for the testing dataset, which represents the existence of an overfitting 
issue. Similar trends can be observed for the cases of the ERT, AdaBoost, and SGBoost but with 
more controlled overfitting. On the other hand, the RF and XGBoost models resulted in lower 
training capabilities, while their testing results represented comparable performance to the ERT 
and SGBoost. This means that the RF and XGBoost techniques are more sensitive to overfitting 
issues. The residual plots for the considered models are provided in Figure 6. In general, it can be 
seen that most of the developed estimation models yielded similar results, with the exception that 
the DT model had the highest distortions in the residuals as compared to others. 
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Fig. 5. Actual versus predicted plots for the investigated models. 
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The error assessment of the fitting rates of the developed prediction techniques is shown in Table 
1. Generally, it can be seen that there are drops in the performances of all models when 
comparing the training to the testing results, which is usually expected but should be controlled 
to avoid overfitting issues. Indeed, the fitting rate of the DT algorithm for the testing subset of 
data was the lowest among others, while the XGBoost reached the highest one. On the other 
hand, the maximum error achieved in the DT model was about 5% lower than the XGBoost 
model. Using the XGBoost model reduced the RMSE value by almost 43%, 22%, and 23% 
compared to the DT, ERT, and SGBoost models. Moreover, the ERT approach had higher 


capabilities than the RF model, which can be attributed to the enhanced randomization abilities 
of the ERT method. 
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Table 1 
Performance of the investigated models in predicting concrete's compressive strength. 
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3.2. Feature importance 


As highlighted previously, feature importance analysis can be used to rank input parameters 
based on their impact on the performance of the machine learning models and identify whether 
or not a specific input variable is necessary for producing the model to be successful. This study 
utilized two approaches to feature importance assessment: drop-out loss and MSE reduction. The 
results of these methods and their comparison are shown in Figure 7 and Table 2, respectively. In 
general, it can be seen that there are some fluctuations in the ranks obtained using each of the 
approaches to feature importance assessment in a particular machine learning model. For 
instance, the drop-out loss in the case of the DT model ranked the water as the fourth while the 
MSE reduction ranked it as the third among the input parameters. Additionally, by comparing the 
results of the machine learning models for the drop-out loss case, each one of the estimation 
techniques has its own ranking. In contrast, the ranks obtained using the DT and RF cases for the 
MSE reduction approach were identical, and the AdaBoost and SGBoost outcomes in that 
method were the same even though their coefficients of determination are different. This 
observation proves the known fact that it has a lower sensitivity as compared to the drop-out loss 
approach. 
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Fig. 7. Feature importance scoring for input parameters of the machine learning models. 
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On the other hand, it can be seen that using feature importance to conduct a parametric 
assessment of the concrete constituent materials is somehow misleading. It can be understood by 
comparing the knowledge available in the literature to the results presented in Table 3. For 
example, in most machine learning models, the fly ash ranked as the least essential variable, 
while blast furnace slag was ranked as the third or fourth in some cases. However, it was 
presented in many studies that the fly ash causes a similar or higher impact than the blast furnace 
slag [59,60]. 


Table 2 
Comparison of drop-out loss and MSE reduction approaches to feature importance analysis. 


Drop-Out Loss 


prop -Outboss 
_—_____} Pr | Rr} RET | Adsboost} SoBoost | Noose 
a A 
Age 


Cement 

Coarse Aggregate 

Fly Ash 

Water | 858 | 1966 | rai | x61 | 1900 | 304 | 
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Table 3 
Ranks of the input features using machine learning techniques. 
Drop-Out Loss 


Rank DT RF ERT AdaBoost SGBoost XGBoost 
1 Cement Age Age Age Age Age 
2 Age Cement Cement Cement Cement 
3 Blast Furnace Slag] Blast Furnace Slag| Blast Furnace Slag Water Blast Furnace Slag Water 
4 Water Superplasticizer | Blast Furnace Slag Water Blast Furnace Slag 
5 Superplasticizer | Superplasticizer Water Fine Aggregate Superplasticizer Fine Aggregate 
6 Fine Aggregate Fine Aggregate Fine Aggregate Superplasticizer Fine Aggregate | Coarse Aggregate 
7 Coarse Aggregate | Coarse Aggregate Fly Ash Coarse Aggregate | Coarse Aggregate | Superplasticizer 
8 Fly Ash Fly Ash Coarse Aggregate Fly Ash Fly Ash Fly Ash 

MSE Reduction 

Rank DT RF ERT AdaBoost SGBoost XGBoost 
1 Age Cement Cement Age 
2 Cement Age Age Cement 
3 Water Water Water Water 
4 | Blast Furnace Slag | Blast Furnace Slag| Blast Furnace Slag} Blast Furnace Slag | Blast Furnace Slag Fly Ash 
5 Superplasticizer | Superplasticizer | Superplasticizer | Superplasticizer | Superplasticizer | Superplasticizer 
6 Fine Aggregate Fine Aggregate Fine Aggregate Fine Aggregate Fine Aggregate | Blast Furnace Slag 
7 Coarse Aggregate | Coarse Aggregate Fly Ash Coarse Aggregate | Coarse Aggregate | Fine Aggregate 
8 Fly Ash Fly Ash Coarse Aggregate Fly Ash Fly Ash Coarse Aggregate 


3.3. Partial dependence analysis 


Indeed, the partial dependency plots represent the marginal influence a certain input parameter 
has on a machine learning model's outcome and illustrate whether the interaction between the 
target and a feature is linear, monotonic, or more complicated. Moreover, it can answer questions 
on how the model's estimation results vary for a feature so that the impact of the predictor is 
clearly stated. The results of this analysis are shown in Figure 8 for the DT and RF models, 
Figure 9 for the ERT and AdaBoost ones, and Figure 10 for the SGBoost and XGBoost ones. It 
can be seen that the trends in all machine learning techniques are primarily similar in terms of 
positive and negative impacts. Generally, the cement, blast furnace slag, and superplasticizers 
positively impacted the concrete's compressive strength in all the investigated models with an 
average peak partial dependence of 30%, 10%, and 5%, respectively. Additionally, an exception 
to these observations is obtained from the XGBoost model for the case superplasticizer, which 
positively impacted specific content and then started reducing considerably. It simulates the 
practical scenario presented in the literature in which a high dosage of chemical admixtures 
harms the strength of concrete [61]. In contrast, the water content had a negative influence on all 
models. 


Y. Alzubi et al./ Journal of Soft Computing in Civil Engineering 6-3 (2022) 39-62 


DT 


Cement 


80 - 
g 
§ 60- 
HS 
a 
8 49- 
3 
e 
é 
\ 
600 
Cement (kg) 
Fly Ash 
10 - 
8- 
64 
g 
5 4- 
3 
2 
3 
& 
rot 
=z 
2 
é 
=43 
~6- 
0 25 50 75 100 125 150 175 200 
Fly Ash (kg) 
Superplasticizer 
25 - 
20 - 
re 
§ 15- 
H 
2 10- 
a 
a 
2 
é 
-s- 
0 5 10 15 2 2 30 35 40 
Superplasticizer (kg) 
Fine Aggregate 
10- 
¢ 
2 
5 
3 
2 
3 
3 
8 
B _10- 
é 10 
© -15- 
~20- 
600 650 700 750 800 850 900 950 1000 
Fine Aggregate (kg) 


Blast Furnace Slag 


35 - 


30- 


Partial Dependence 
ee NN 
ows GDS 
re ee er eer 


-5- 
0 50 100 150 200 250 300 350 400 
Blast Furnace Slag (kg) 
Water 
10- 
g o- 
2 
2 
z 
2 -10- 
s 
a 
3 
£ -20- 
é 
~30- 
405 ‘ ‘ ‘ ' ‘ ' ' 
100 125 «4150 175 200 225 250 275 300 
Water (kg) 
Coarse Aggregate 
15 - 
10- 
¢ 
2 
3 354 
2 
5 
Fy 
a 0 
3 
2 
& -57 
-10- 
800 850 900 950 1000 1050 1100 1150 1200 
Coarse Aggregate (kg) 
Age 
80 - 
70 - 
g 60- 
§ 50- 
ro 
& 40 - 
6 
B 30- 
€ 
€ 20- 
10 - 
ah hh 
i i i j : A ‘ ' 
0 50 100 150 200 250 300 350 400 
Age (days) 


Cement 


win 
o 6 


Partial Dependence 
> 
é 


30- 
20- 
10 - 
ty) i i ' 
100 200 300 400 500 600 
Cement (kg) 
Fly Ash 
a4 
2- 
rm 
€ 
3 1- 
2 
5 
a 
6 
3 
Ea 
-24 
0 25 «50 75 100 125 150 175 200 
Fly Ash (kg) 
Superplasticizer 
17.5 - 
15.0 - 
9 
& 12.5 - 
5 
i 10,0 - 
8 75- 
' ' ' : ' ' ‘ ‘ ‘ 
t) 5 10 8615062025835 
Superplasticizer (kg) 
Fine Aggregate 
75- 
5.0- 
25- 


g 
g 
2 
3 
g 
-25- 
8 
3 
t 
& 


Partial Dependence 


55 


Blast Furnace Slag 


NooON 
os a 
ri 1 


Partial Dependence 
~ 
ir 
, 


De D 7 ee ree eee ‘ 
0 50 100 150 200 250 300 350 400 
Blast Furnace Slag (kg) 
Water 
o- 
-5- 
-10- 
-15- 
~20- 
-25- 
100 125 150 175 200 «225 «-250«275 300 
Water (kg) 
Coarse Aggregate 
re 
34 
¢ 
2 
§ 
3 
2 
go 
a 
a 
B-a- 
=e 
é 
-6- 
-8- 
800 850 900 950 1000 1050 1100 1150 1200 
Coarse Aggregate (kg) 
Age 
70 - 
60- 
$ 50- 
5 
3 
§ 40- 
g 
a 
3 
= 
5 
é 


-10.0- 
-125- 7 
‘ ‘ ‘ ‘ ‘ ‘ ‘ ' ' 7 7 i 7 7 7 ‘ 
600 650 700 750 800 850 900 950 1000 oO 50 100 150 200 250 300 350 400 
Fine Aggregate (kg) Age (days) 


Fig. 8. Partial dependence curves for the DT and RF models. 
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Fig. 9. Partial dependence curves for the ERT and AdaBoost models. 
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Fig. 10. Partial dependence curves for the SGBoost and XGBoost models. 
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Additionally, the influences of the contents of fine and coarse aggregates and fly ash fluctuated 
in most models. However, research in the literature concluded that the high content of fly ash 
harms the strength of the concrete. This point was only obtained in the case of the XGBoost 
model, while other approaches showed almost no impact at will. 


4. Conclusion 


In conclusion, this study has focused on evaluating the application of machine learning 
techniques for conducting a parametric assessment to evaluate the influence of concrete mixture's 
consistent materials on its compressive strength. 


e Within the research context, six different machine learning models were developed and 
assessed their performance. After that, two different methods were used in conducting a 
parametric assessment in the artificial intelligence environment. The first one was the 
feature importance which tries to rank the inputs of the estimation model for their 
contribution to the model's accuracy. In contrast, the second one illustrates the effect of 
variation in each input feature on the model's outcomes. 

e In general, the study results have shown that using the feature importance to understand 
which parameter affects the strength of concrete is somehow misleading in the drop-out 
loss, and MSE reduction approaches since their outcomes did not match the expectations 
based on the knowledge available in the literature. 

e Nevertheless, the partial dependence plots provide a clear idea about the influence of the 
value of each feature on the prediction outcomes, especially in the case of the XGBoost, 
in which the results were the closest to the conclusions of previous research. 

e Accordingly, this study recommends utilizing partial dependence analysis with the 
XGBoost model to evaluate the constituent materials' influence on the compressive 


strength of concrete. 


Data availability 


The raw data that were used to support the findings of the study can be obtained from the 
following link: "ttps://archive.ics.uci.edu/ml/datasets/concrete+compressivet+strength" 
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